Learning by Doing and the Value of Optimal Experimentation

Research on the implications of learning-by-doing has typically been restricted to specifications of the agent's decision problem for which estimation and control can be treated separately. Recent work has provided the limit properties of beliefs and actions for learning problems under more general conditions, for which experimentation is an important aspect of optimal control. However under these conditions the optimal policy cannot be derived analytically, because Bayesian learning introduces a nonlinearity in the dynamic programming problem. This paper utilizes numerical algorithms to characterize the optimal policy function for such a general learning-by-doing problem. In contrast to previous work on calculating such policies, we find that the optimal policy incorporates a substantial degree of experimentation under a wide range of initial beliefs about the unknown parameters. Dynamic simulations indicate that optimal experimentation dramatically improves the speed of learning and the stream of future payoffs. Furthermore dynamic simulations reveal that a policy, which separates control and estimation and does not incorporate experimentation, frequently induces a long-lasting bias in the control and target variables. While these sequences tend to converge steadily under the optimal policy, they frequently exhibit non-stationary behavior when estimation and control are treated separately. JEL No.: C44,C60, D81, D82